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Abstract 

This report shares the findings and lessons learned from an investigation into the economics of disaggregated 
models for assessing and accrediting informal learners undertaking post secondary education. It presents 
some key economic and governance challenges for universities to consider in implementing OER assessment 
and accreditation policies. It also includes discussion of findings from a small-scale survey conducted by two 
of the authors on perceptions, practices and policies relating to openness in assessment and accreditation 
in post secondary institutions, with a particular focus on the OER universitas (OERu) concept. 
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Introduction 

The concept of “openness” is arguably the most persistent and controversial educational innovation 
of recent years, provoking the potential for important change in post secondary education, world¬ 
wide. The two key concepts contained within the movement are Massive Open Online Courses 
(MOOCs) and Open Educational Resources (OER). Following an explanation of these important 
phenomena, this paper explores the key issues of assessment and accreditation that are raised by 
the notion of openness in learning. It is based on a report that examines potential models to address 
the learner assessment, certification and accreditation issues for learners participating in MOOCs 
as well as in other formal and informal learning contexts using OER at the higher education level 
(Conrad, Mackintosh, McGreal, Murphy & Witthaus, 2013). We focus here on the organisation and 
mandate of the Open Education Resource universitas (OERu), a global consortium of post second¬ 
ary institutions whose collaboration around the assessment and accreditation of learners studying 
online and using open educational resources (OER) facilitates greater access and flexible learning 
paths for learners. 

MOOCs have experienced a rapid growth in interest, often negative, since their inception in 2008. 
While permitting access to the same course for large numbers of learners in many countries, the 
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introduction of MOOCS has given rise to the need to address issues of learner assessment and 
accreditation. Learners who access digital learning content via the Internet and acquire knowledge 
and skills either formally or informally, alone or in groups, cannot readily have their learning assessed, 
and are consequently unable to receive appropriate academic recognition for their efforts (Taylor, 
2011; Mackintosh, McGreal & Taylor, 2011). This critical issue is the sticking point for many educa¬ 
tors and as such has attracted attention both from institutions leading the OER movement and from 
its critics (Phelan, 2012; Yuan, MacNeill & Kraan, 2008; Olcott, 2012). 

OER have provoked similar controversy, and institutional participation in the development and 
use of OER has been patchy across the globe. In many parts of the world, few institutions indicate 
that they either produce or use OER, and in the regions where production and usage of OER are 
higher, there is little coordination between institutions and generally not much governmental support 
to sustain OER activities. (See, for example, the early findings from the POERUP project, Bacsich 
et al., 2013.) Even fewer institutions have implemented open courses for assessment and accred¬ 
itation (Conrad et al., 2013). In spite of resistance, the open movement continues to expand as 
evidenced by the growth of online OER repositories and the ongoing swirl of attention to MOOCs 
as well as the establishment of collaborative initiatives based on the use and reuse of OER. 

The Study 

In 2012, two of the authors of this paper developed a quantitative research survey to investigate 
the perceptions, practices and policies of post secondary institutions worldwide, toward openness 
in assessment and accreditation. A sample of 110 individuals representing 83 education institutions 
was obtained over a three-month period from June to August 2012. The survey was programmed 
and hosted online, with invitations to participate disseminated through social media channels and 
discussion forums aimed at educators interested in Open Educational Resources (OER). Invitations 
were also emailed to individual representatives of higher education organisations that were known 
to be engaged with OER assessment and accreditation initiatives such as the OER universitas 
initiative (previously known as the OER university). 

Responses originated from 29 countries including the regions of Western Europe (United Kingdom 
and Ireland) (40%, n=44), North America (17%, n=19) and Asia (14%, n=15) followed by Australia 
or New Zealand (9%, n=10), Africa (9%, n=9), South America (7%, n=6), and other parts of Europe 
(6%, n=7). A range of education institutions was represented, with universities forming the largest 
cohort (64%, n=68), followed by public organisations, not for profits or other non-teaching organisa¬ 
tions (27%, n=29), polytechnic universities or institutes of technology (including TAFE) (9%, n=10), 
secondary institutions (8%, n=9), vocational training providers (6%, n=6) and 3 year community 
colleges (2%, n=2). Representation from a range of levels within organisations was obtained, includ¬ 
ing practitioners (48%, n=53) including lecturers, teachers, trainers, educational designers, and 
project workers to researchers (25%, n=14), managers (8%, n=9), senior managers (14%, n=15) 
and executive management (11%, n=12). 

The Open Education Resource universitas (OERu) 

The OERu is a collaborative partnership currently comprising 30 partner institutions. The network 
includes both nationally accredited universities, colleges and polytechnics and publicly-funded 
organisations (Mackintosh, Taylor & McGreal, 2011). OERu’s current geographic spread of institu¬ 
tions includes Australia, New Zealand, Canada, the USA, England, Ireland, Spain, South Africa, 
India and the South Pacific. The initiative is coordinated by the OER Foundation, which is an inde¬ 
pendent, not-for-profit organisation that works internationally to support the mainstream adoption of 
OER into the formal education sector (Mackintosh, 2012). 
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Mackintosh, Taylor and McGreal (2011) described the aim of the OERu as providing free educa¬ 
tion to learners worldwide using OER as learning materials, thus providing pathways to enable 
learners to gain credible qualifications from government-recognised or accredited educational 
institutions. The OERu concept is rooted in the notion of community service and outreach, and 
institutions that are members of the OERu network have committed to developing a “parallel learn¬ 
ing universe” that will augment and add value to traditional delivery systems in post secondary 
education. The ultimate vision of the OERu is to provide free learning opportunities on a massive 
scale for learners who lack the financial means to access traditional higher education (Conrad 
etal., 2013). 

One of the cornerstones of the OERu philosophy is that the components of higher education that 
are traditionally packaged together in a single institution can be disaggregated and provided by 
different institutions. Anderson and McGreal (2012) suggest that discount service models may 
become attractive, suggesting that the Open Educational Resources universitas initiative could 
support disaggregation as a disruptive model. Murray and Friesen (2011) explain that in traditional 
education models, learners enrol with a single institution and expect that institution to provide the 
teaching, the content, the assessment and the eventual accreditation. In the OERu’s disaggregated 
model, “the basic elements of education, traditionally conceived, are redefined as placeholders 
and are opened up to substitution and disaggregation. Any student can study any content, supported 
in any number of instructional arrangements” (Murray & Friesen, 2011, p. 4). This “disaggregation” 
distinguishes the OERu model from other open learning models such as MOOCs and gives rise to 
the discussion of assessment and accreditation potential that follows. 

Accreditation and assessment in post secondary education: Issues and approaches 

The assessment of learning and its resultant accreditation toward a credential presents a major 
hurdle to the integration of open learning with formal learning (Conrad, 2013; Friesen & Wihak, 
2013) and provided the raison d’etre to the study that backgrounds this paper. 

Academic assessment remains, universally, the privilege and purview of individual post secondary 
systems. While several jurisdictions have developed and implemented national accreditation frame¬ 
works, notably the UK and Australia, international accreditation and assessment services are not 
currently available. It is the premise of this paper that developing a robust system that can service 
thousands or even hundreds of thousands of learners internationally would change the dynamic of 
access to post secondary education for learners. The necessary systems technology is already 
available within post secondary institutions and would remain within their control: payment systems, 
content management systems, automated examination applications, and online invigilation. 

There are two main types of assessment or accreditation relatively common at the post second¬ 
ary level of education outside of traditional classroom-based assessment: the Recognition of Prior 
Learning (RPL) and post secondary credit transfer, where formal credentials obtained at another 
institution are transferred into learners’ programs at their current or home institution. RPL is often 
referred to as Accreditation of Prior and Experiential Learning (APEL), Prior Learning Assessment 
(PLA, primarily in the USA), and Prior Learning Assessment and Recognition (PLAR, primarily in 
Canada) as well as several other terms. 

RPL processes exist at post secondary institutions in many countries. Research shows that 
the use of learning portfolios in RPL is very popular, followed by examinations that allow learners 
to challenge-for-credit through assignments, examinations, interviews, courses, tutorials, demon¬ 
strations, self-assessment, external evaluations, essays, face-to-face or online workshops, and a 
variety of other instruments (Conrad & McGreal, 2012). Proficiency and knowledge acquisition can 
be demonstrated through the use of a single assessment methodology or a combination of the 
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above. The least frequently used and available (though it could possibly be the most effective from 
the perspective of learners) is the option to write a challenge examination or engage in some other 
form of challenge-for-credit. Other methods of RPL are much more resource intensive, requiring 
staff dedicated to spending significant time with prospective students. RPL-supportive institutions 
have developed resources and structures that cover most aspects of accreditation and assessment, 
including policy, research, repositories and experience with licensing. 

Credit transfer refers to the willingness of institutions to grant credit to students who have taken 
courses at other institutions. While easier and less labour-intensive than RPL, credit transfer-in 
those institutions that permit it- is sometimes problematic for students. In North America, with its 
standard three credits for a one-semester course, credit transfer is easier to implement and there¬ 
fore reasonably common, especially in the first two years of post secondary education. However, 
this is not the case in jurisdictions outside of the USA and Canada, where many institutions are 
reluctant to accept transfer credits and the majority of students are restricted to taking all of their 
courses at one institution. 

The MOOC phenomenon has opened up interest in the possibility of alternative assessment and 
although many institutions around the world are considering these alternatives, breaking down 
institutional silos continues to present a major hurdle in the “cottage industry” of post secondary 
education, a hurdle that must be cleared before large-scale OER-based courses can be put in place. 
Using RPL for assessment could possibly offer a solution (Camilleri & Tannhauser, 2012; Conrad, 
2013). 

In a scalable, open environment, the necessary unbundling of services to separate assessment 
and accreditation from teaching and institutional support can be much easier using OER-based 
courses rather than commercial content. Scalability is problematic for initiatives that rely on com¬ 
mercial content that is restricted by technological protection measures and restrictive licensing. 
Because of the ease of copying, adapting and otherwise reusing OER, OER-based initiatives can 
be scaled up and made freely available in different jurisdictions and institutions. 

Learning the hard lessons: Issues around and barriers to implementing OER 

Murphy and Witthaus’ (2012) findings uncovered issues and barriers to the implementation of OER 
in post secondary institutions. The study also suggested that the notion of disaggregating traditional 
university services, while providing a useful conceptual framework for considering assessment and 
credentialing alternatives in open learning environments, has yet to be fully understood by the wider 
higher education community and has yet to be operationalised in practice at anything approaching 
a meaningful scale. 

Study findings suggest that there is sufficient evidence to justify the unbundling of traditional 
services in order to provide more affordable access to post secondary education and formal aca¬ 
demic credit. A small number of institutions are already utilising these opportunities as part of their 
existing delivery models. For example, the University of South Wales is already implementing RPL 
on a larger scale than most UK universities and is planning to expand the scope of its accreditation 
activities in partnership with other institutions in the OERu (Witthaus, 2013). Even if we consider 
the commitment of only the 30 OERu member institutions to establishing a “world OER credit bank” 
and “specifying what credit they are willing to accord those who successfully complete the learning 
outcomes associated with [the OER]” (OERu, 2011, p. 20) it is reasonable to assume that the inven¬ 
tory of full programmes of study that will be available under an assessment-only model will grow in 
the coming years. For example, in 2012, Otago Polytechnic announced that the new Graduate 
Diploma in Tertiary Education approved by the New Zealand Qualifications Authority will be based 
entirely on OER and will also cater to assessment-only options for prospective learners (Otago 
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Polytechnic, 2012). Also in 2012, the South African Ministry of Higher Education and Training stated 
that “collaborative development of high quality learning resources made available as Open Educa¬ 
tional Resources (OER) provides the possibility both for increased access and quality and lower 
unit costs due to reduced duplication and greater usage.” (DHET, 2012). Additionally, the DHET 
(Department of Higher Education and Training) has more recently declared its intention to work 
“toward creating a post-school distance education landscape based on open learning principles” 
(DHET, 2013). 

Mainstream assessment approaches that are used for summative assessment and credentialing 
in traditional models, such as examinations, tests, and, increasingly, portfolios, can be reused effec¬ 
tively within a disaggregated system. Challenge-for-credit examinations present an alternative within 
disaggregated systems that enable learners to prove mastery of a set of learning outcomes by 
sitting a challenge exam at a reduced fee compared to the full-tuition price of a course (Conrad 
et al., 2013). And while portfolio assessment processes can accredit knowledge gained from prior 
experience toward learners’ credentials, the labour intensive nature of this process may be less 
suitable for reuse within disaggregated systems. The study did not identify any material policy bar¬ 
riers for assessment and credentialing that would curtail the implementation of a disaggregated 
model, although in the UK there is a perception that the Quality Assurance Agency’s (QAA) strict 
rules for “collaborative provision” could make it difficult for UK institutions to operate in this way 
(Bird & Witthaus, 2012, p. 3). Elsewhere in the world, it appears that the majority of post secondary 
institutions would be able to implement assessment-only models within existing policy frameworks. 
However, minor adaptations and refinements to operational management and processes may be 
required. For example, student administration and processing of payments within a disaggregated 
model would require the accommodation of individual payment-per-assignment submitted for 
assessment. Such a system would also create a labour-intensive process that may be problematic 
within traditional models that require payment by learners prior to the recording of credit on 
university transcripts. 

The practice of providing free online learning opportunities and corresponding solutions for assess¬ 
ment, certification and credentialing services will continue to evolve. Consider, for example, that the 
American Council of Education (ACE) has begun a project to explore whether credit recommenda¬ 
tions from the Council can provide a viable pathway for accreditation of MOOC learning (Fain, 2012). 
This forms part of a $3 million investment by the Bill and Melinda Gates Foundation in several 
MOOC-related investigations including the Association of Public and Land Grant Universities (APLU) 
that are designed to create an interactive learning consortium to study the potential of MOOCs for 
public community colleges and universities and the Ithaka Strategic Consulting and Research group 
that will work with the University System of Maryland to test and study the use of MOOCs across 
their system (Fain, 2012). 

Moreover, following the adoption of the 2012 Paris OER Declaration at the UNESCO OER 
World Congress, governments of member states of the United Nations will be encouraged to adopt 
policies that require teaching and learning materials produced from public funding to be released 
under open content licenses (UNESCO, 2012). This action may foster new policy imperatives 
for publicly funded universities and colleges to encourage them to diversify formal assessment 
and credentialing protocols for their citizens using OER courses as they can provide cost-effective 
pathways for widening access to post secondary education. 

Although research on OER initiatives is contributing new knowledge for both researchers and 
practitioners, barriers to forward-movement in “openness” were also identified in the Murphy and 
Witthaus study (2012). The following section discusses barriers encountered and lessons learned 
to date. 
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Current institutional processes for international credit transfer and course articulation are idiosyn¬ 
cratic, usually dependent on institutional policy, with a lack of standardisation when working across 
regional and national borders (Conrad & McGreal, 2012). As such, current course articulation pro¬ 
cesses are not well suited to the recognition and credentialing of OER learning. This is potentially 
the most significant policy barrier for scalable implementations of assessment and accreditation for 
official recognition of learning on a large and global scale. Current institutional processes often result 
in unnecessary duplication and inefficiencies that hinder the cost-effective implementation of assess¬ 
ment and credentialing, which could be avoided through carefully structured collaboration around 
OER (see, for example, the case studies described by Lane, 2012). 

Other significant barriers to OER identified by faculty and administration include fear of change, 
confusion over copyright issues and the use and reuse of OER, concerns regarding the effort 
required for implementation of OER initiatives, and the possibility of conflict with commercial pub¬ 
lishers and other special interest groups (Murphy & Witthaus, 2012). Perceived barriers to the kind 
of collaboration proposed by the OERu are also numerous, and include concerns about the role of 
national quality assurance bodies, a belief that students might not get sufficient support, concerns 
about the “true” cost of collaboration around OER (in financial terms), and some scepticism around 
the philanthropic motives of the OERu (Bird & Witthaus, 2012, p. 14). However, the greatest barriers 
to participation in open assessment and accreditation practices are the lack of availability of com¬ 
mitted staff members to support such activities and the potential costs of redeveloping courses as 
OER (Murphy & Witthaus, 2012). Lack of support for OER-based courses from senior management 
is also a substantial concern. 

These “barriers” can be countered by incentives such as the low cost of entry and use of OER; 
minimal or non-existent licensing requirements; the ability to localise and update the content 
and make other changes without restriction; greater acceptance by students; organisational leader¬ 
ship by educators; and the potential to increase the institution’s ability to serve a greater number 
of international students and raise the profile of the institution in the global higher education 
community. 

Within institutions, key factors for the success of open assessment and accreditation implemen¬ 
tation appear to be a reliance on a strong base of support within the institution—both in terms of 
leadership and resources—and an existing culture of openness that includes policies and practices 
around the creation and use of OER (Murphy & Witthaus, 2012). Policies that enable either open 
access or recognition of prior learning via credit transfer or RPL are also important. Institutions 
that already have these features in place are likely to be in the best position to implement assess¬ 
ment and accreditation of OER-based learning services, and, as such, could provide models for 
other organisations that would like to participate in collaborative open education, assessment and 
credentialisation initiatives in the future. 

Considerations for future research 

Several areas for future research become obvious when the barriers and hurdles facing further 
implementation of OER are considered: 

Establishing cross-border systems of articulation and transfer. As outlined above, the current state of 
idiosyncratic course articulation processes does not foster credentialing processes within disaggregated 
systems for OER learning. Further research is required to explore alternatives for cross-border credit 
transfer and course articulation in an open and disaggregated manner. As institutional accreditation is 
closely associated with quality assurance mechanisms, this research will need to consider corresponding 
implications for quality assurance processes in a disaggregated system. 
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Applying RPL processes to a disaggregated model of learning. The majority of OERu institutions 
and practitioners consider RPL methods and approaches to be the main vehicle for the assessment 
and credentialing OER learning in a disaggregated model (Murphy & Witthaus, 2012). Although a 
preliminary analysis of the costs to students for RPL are expensive when compared to other cre¬ 
dentialing alternatives, for example, automated assessment and course assessment packages, 
these alternative methods should not be considered as substitutes for RPL because portfolio assess¬ 
ment offers meaningful opportunities for the assessment and accreditation of learning acquired 
outside of conventional course delivery (Camilleri & Tannhauser, 2012; Conrad, 2013; Friesen & 
Wihak, 2013). While for many learners, RPL is the only option available to gain formal recognition 
of their learning, there may be specific RPL processes which could be reused, repurposed and 
repackaged to augment and support assessment and credentialing processes for OER learning 
in a disaggregated system. To investigate these possibilities, further research focusing on the 
following areas of RPL is recommended: 

• Activity-based costing analysis of RPL processes to identify cost-behaviours of appropriate 
RPL activities; 

• Investigation of possible solutions for cost-effective and scalable RPL processes when 
working with large numbers of learners; and 

• Analysis of alternatives for packaging RPL assessment and credentialing processes and 
corresponding pricing for sustainable operations in a disaggregated system. 

Conclusion 

Academic boards and senates at many universities are reluctant to reuse open-licensed courses 
and their corresponding assessments, even though those materials have been formally approved 
by another accredited university and even though these open courses can be adapted locally at no 
cost and offered in parallel with existing courses in order to diversify curriculum at the home 
institution. This reticence may prove to be short-sighted and poor business strategy. As they so 
often do -think Coursera- the commercial sector will be quick to appreciate the business value of 
building assessment and credentialing strategies from assets which do not require any upfront 
investment. And governments hard-pressed to reduce fiscal deficits may consider alternatives for 
a more cost-effective post secondary sector that recognises the benefits of favouring the disag¬ 
gregation of traditional university services. 

The formal post secondary sector has a unique opportunity to take a leadership role in determin¬ 
ing its own futures because the token esteem of a university credential continues to be highly valued 
by both society and economy, as Brown and Duguid (1996) outlined: 

in our highly commodified society it is naive to believe that access on its own is enough. Those who 
have the label but not the experience present one problem. But those who might have the experience 
but not the label face another. Experience without a formal representation has very limited exchange 
value—as those whose only degree is from the university of life well know. (p. 10) 

Universities, as institutions, are understood to have the requisite knowledge and experience and 
have long enjoyed the trust of society in the process of accrediting the formal learning of those 
seeking formal academic credentials. The disaggregation of assessment and credentialing services 
for OER learning can provide a viable pathway for more affordable access to post secondary 
education and formal academic recognition while simultaneously continuing to serve universities’ 
core academic missions of disseminating knowledge and engaging in community service. 
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